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I.  Introduction 


Various  microsensor  networks  sueh  as  those  exemplified  by  Bluetooth  teehnology  have 
emerged,  with  further  miniaturization  toward  the  nanoscale  by  utilizing  eommereial 
smarteard  teehnology,  for  military  and  other  applieations.  The  problem  of  information 
security  is  of  central  importanee  in  both  of  these  types  of  short-range  wireless  radio 
networks,  whieh  share  the  common  characteristics  of  limited  resourees  for 
eommunieation  and  eomputation.  The  individual  sensors  need  to  be  able  to  authentieate 
one  another  and  to  eommunieate  seeurely  at  Mbps  rate  up  to  100  m  distanee,  and  perhaps 
oecasionally  with  a  larger  base  station,  in  order  to  perform  certain  joint  signal  processing 
and  monitoring  operations,  to  hold  some  data  in  memory,  and  to  deliver  them  to  their 
final  destination.  One  overall  goal  would  be  to  minimize  the  power  eonsumption  in  order 
to  maximize  the  lifetime  of  the  sensor  network  operation  and  the  amount  of  useful 
proeessing.  Power  minimization  has  been  an  important  eonsideration  in  computation  for 
some  time.  The  Crusoe  proeessor  supposedly  saves  power  by  having  more  functions 
implemented  in  software,  and  reversible  logie  may  hold  some  promise  in  the  future  for 
speeial  hardware  power  reduetion.  Sueh  general  future  possibilities,  while  useful,  are  not 
speeifically  tuned  to  the  problem  of  seeurity. 

The  objeetive  of  this  projeet  is  to  develop  eryptographie  teehniques  that  aehieve  a  high 
level  of  performanee  in  terms  of  both  seeurity  level  and  eneryption/deeryption  rates  while 
at  the  same  time  offering  a  new  level  of  power  effieieney  for  energy-eonstrained  sensors. 
They  would  be  useful  with  any  hardware,  ineluding  the  Crusoe  proeessor  and  reversible 
logie  when  available.  Many  eryptographie  techniques  have  been  developed  in  the  similar 
eontext  of  eellular  phones,  Bluetooth,  and  smarteard  applieations.  Some  eonsiderations 
of  power  effieieney  are  given  in  the  first  two  of  these  teehnologies,  whieh,  however,  only 
involve  linear  feedback  shift  register  and  the  new  bloek  eipher  SAFER+  whose  seeurity 
levels  are  not  very  high  [Seh96],  [MOV97].  Energy  has  not  been  an  issue  in  smart  eards, 
whieh  are  powered  mainly  by  maehine  terminals.  An  important  eonsideration  in  our 
project  is  to  integrate  the  forward  error  eontrol  necessary  for  reliable  eommunieation  to 
the  eryptographie  mechanisms,  which  are  often  eompromised  by  eommunieation  ending  - 
-  a  good  example  of  sueh  a  eompromise  is  found  in  the  eellular  IS-95  CDMA  protoeol. 
We  have  developed  entirely  new  signal/noise  eryptographie  teehniques  as  well  as  new 
stream  eipher  algorithms  to  solve  this  problem.  We  have  also  applied  various  standard 
and  novel  VESI  power  minimization  teehniques  to  modify  various  eryptoalgorithms 
whieh  are  eompared  in  an  easy-to-use  form. 

There  are  three  major  funetions  in  a  eryptographie  system:  key  management, 
authentieation  and  eneryption.  The  algorithms  we  ehoose  to  study  and  develop  inelude 
the  standard  ones  based  on  shared  seeret  key,  dual  publie/private  keys,  ones  for  random 
number  generations,  for  stream  eiphers,  and  new  ones  based  on  eommunieation  signaling. 
They  ean  be  used  as  modules  in  a  variety  of  eneryption/deeryption,  key  distribution, 
message  authentication,  and  identification  (user  authentication)  protocols.  It  is  not  our 
purpose  to  investigate  and  develop  sueh  protocols,  although  they  would  also  have  effeet 
on  power  assumption,  exeept  to  note  the  following.  In  many  applieations  of  sensor 
networks,  there  is  no  need  to  have  separate  keys  for  different  cells  or  groups  for  many 
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cryptographic  functions,  such  as  encryption  from  one  point  to  a  distant  point.  Use  of  the 
same  key  would  greatly  reduee  the  power  eonsumed  by  deeryption/re-eneryption  when 
hopping  from  one  eell  to  another.  On  the  other  hand,  the  frequent  change  of  session 
keys,  while  power  eonsuming,  enhanees  the  security  level  and  eould  lead  to  a  final  saving 
of  total  power  if  an  otherwise  less  seeure  but  also  less  power  eonsuming  algorithm  ean  be 
used  instead. 

The  project  consists  of  three  major,  inter-related  tasks  and  one  additional  task 

■  Task  A:  Novel  Stream  Ciphers 

■  Task  B;  New  Spread  Signal  Cryptography 

■  Task  C:  Power  Predietion  and  Reduetion 

■  Task  D;  Image  and  Video  Proeessing  and  Communieations. 
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II.  TASK  A:  Novel  Stream  Ciphers 

Introduction 

In  this  project,  we  address  the  problem  of  designing  and  implementing  efficient 
cryptographic  protocols  and  algorithms  for  low  power  devices.  The  goal  is  to  ensure  that 
user  authentication,  communication  security  and  key  management  problems  can  be 
handled  efficiently  over  low  power  devices.  To  this  end,  we  consider 

1.  the  design  and  implementation  of  Software  Based  Stream  Cipher  (SSC2)  on 
small  handheld  devices,  such  as  Palm  Pilots  and  iPaqs. 

2.  performance  comparison  of  SSC2  with  other  software  based  encryption 
algorithms. 

3.  implementation  of  Mutually  Authenticated  Key  Exchange  Protocol  using  SSC2 
for  data  integrity  and  privacy. 

4.  the  implementation  of  a  group  key  management  system  to  manage  dynamic 
changes  of  membership  within  a  group. 


Software  Based  Stream  Cipher 

Chan  and  Zhang  [1]  first  proposed  the  design  of  Software  Based  Stream  Cipher,  SSC2,  in 
2000.  To  ensure  computational  efficiency  in  software,  we  design  the  cipher  based  on  a 
byte -based  (or  word-based)  linear  feedback  shift  register,  together  with  simple  nonlinear 
functions  such  as  modulo  add,  carry  and  shifts.  Figure  1  provides  a  schematic  design  of 
the  stream  cipher.  The  security  of  the  cipher  against  currently  known  attacks  has  been 
analyzed  and  published  in  [1,2].  Other  cryptographers  [3]  have  also  studied  the 
vulnerability  of  the  cipher. 


Figure  1:  Schematic  Design  of  SSC2 
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The  performance  of  the  cipher  on  low  power  devices,  such  as  Palm  OS,  was  compared 
with  other  software  based  ciphers  (given  in  Table  2),  in  particular,  the  Alleged  RC4  and 
Software  Encryption  Algorithm  (SEAL).  It  is  observed  that  SSC2  outperforms  both 
algorithms.  The  results  are  documented  in  [4]  attached  in  Appendix  A. 


Table  1  Performance  Comparison  of  Software  Based  Encryption  System  over  Palm  OS 

(Throughput  in  Bytes  per  second) 


"''"'■'-J^ata  size 
Algorithm  ^ — 

2  KB 

50  KB 

4  MB 

SSC2 

32,604 

35,804 

35,501 

ARC4 

30,768 

32,100 

31,699 

SEAL 

2,469 

28,723 

51,396 

Mutually  Authenticated  Key  Exchange  and  Key  Management 

Since  SSC2  is  a  symmetric  key  cipher,  the  efficiency  of  key  management  has  to  be 
considered.  We  first  consider  client-server  model  where  the  client  is  low  powered  and 
requires  lightweight  cryptographic  algorithms,  the  server  may  be  a  base  station  with 
battery  and  computational  power  equivalent  of  a  desktop.  We  implemented  a  mutually 
authenticated  key  exchange  protocol  where  low  power  clients  depend  on  symmetric  key 
operations  based  on  SSC2  and  the  servers  depend  on  public  key  cryptographic 
operations.  Table  3  below  shows  the  performance  comparison  of  the  Server  Specific 
Mutually  Authenticated  Protocol  (SSMAKEP)  and  the  Station  to  Station  (STS)  Protocol 
used  over  the  wired  network. 


Table  2  Performance  Comparison  of  SSMAKEP  and  STS  Protocol 
(authentication  completed  in  msec) 


SSMAKEP 

STS 

Palm  OS  vs. 

Linux 

220  -  260  msec 

738,740  msec 

Linux  vs.  Linux 

17-27  msec 

494  -  515  msec 

Once  the  client  and  server  shares  a  common  key  between  them  through  the  key  exchange 
protocol,  clients  with  small  devices  will  then  communicate  through  a  common  group  key 
provided  by  the  server.  Since  the  group  membership  changes  dynamically,  the  server  has 
to  provide  a  key  update  service  that  clients  can  receive  the  new  group  key  efficiently  and 
securely.  A  Group  Key  Management  Scheme  has  been  implemented  as  part  of  the  effort 
in  the  project. 
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III.  TASK  B:  New  Spread  Signal  Cryptography 


We  have  invented  a  new  teehnique  that  utilizes  proper  signal  modulation  and  ehannel 
noise  but  without  these  weaknesses,  and  also  without  bandwidth  expansion.  The  idea  is 
based  on  analogy  with  quantum  eryptography  [BBA92]  and  ean  be  eonsidered  a 
generalized  form  of  spread  speetrum  —  "spread  signal"  (SSi)  in  the  given  signal  space  for 
which  the  important  general  variable  is  signal  location  in  the  space,  parameterized  by  its 
amplitude  and  phase  in  addition  to  frequency.  The  idea  can  be  applied  to  any  digital 
modulation  format  including  the  usual  phase-shift  keying  (PSK)  and  frequency-shift 
keying  (FSK)  convenient  for  wireless  applications,  as  well  as  amplitude-shift  keying 
(ASK)  that  allows  incoherent  detection  without  bandwidth  consumption.  For 
definiteness  we  introduce  it  in  the  context  of  M-ary  phase-shift  keying  (MPSK) 
modulation  in  which  one  of  M  possible  signals  of  energy  E  lying  on  a  circle  of  a  two- 
dimensional  signal  space,  typically  from  the  two  quadratures  of  a  single-frequency  signal, 
can  be  transmitted  as  indicated  in  Fig.  2.  The  actual  signaling  scheme  to  be  employed  is, 
say,  a  binary  phase-shift  keying  (BPSK)  system  with  M/2,  for  even  M,  possible  pairs  of 
opposite  signals  on  the  circle  selected  out  of  the  M  PSK  signals.  As  depicted  in  Fig.  3a, 
if  the  receiver  needs  only  to  discriminate  between  a  BPSK  pair,  say  in  additive  white 
Gaussian  noise  (AWGN),  the  optimal  decision  can  be  arranged  as  usual  for  equiprobable 
inputs,  with  decision  region  I  for  value  I  and  region  II  for  value  0.  For  any  signal-to- 
noise  SNRb  =  2Eo/No,  one  obtains 

Pe~exp{-SNR}  (1) 

However,  if  the  receiver  has  to  discriminate  a  large  number  of  M  PSK  signals  in  noise 
level  Ni  with  signal  energy  Ei  the  same  error  behavior  (1)  is  obtained  with  a  signal-to- 
noise  ratio  SNRm  =  (4n  /M  )  x  Ei/Ni  for  large  M  >10  as  seen  in  Pig.  3b.  A  small  noise 
N  that  would  not  cause  an  error  in  BPSK  may,  by  pushing  the  signal  for  1  outside  its 
decision  region  (a)  to  the  decision  region  (b)  for  a  0.  Thus, 

SNRm  /SNRb  =  (27r^/M^)  x  (EiNo/EqNi)  (2) 

and  the  receiver  performance  can  be  made  to  be  much  poorer  than  the  BPSK  case  by 
making  M  sufficiently  large  for  any  No,  Ni,Eo,  Ei.  Now  the  transmitter  modulates  a  data 
bit  stream  by  selecting  one  of  the  M/2  BPSK  pairs  for  each  bit  according  to  some  fixed 
rule  shared  by  the  receiver  but  unknown  to  the  eavesdropper  who,  not  knowing  which 
pair  to  use,  would  not  be  able  to  identify  the  signal  in  a  way  similar  to  PN  spread 
spectrum.  In  an  AWGN  channel  or  a  coded  communication  system  where  multipath, 
fading,  and  interference  are  important,  the  error  probability  Pg  can  be  made  exponentially 
small  in  the  SNR  as  in  (1).  Prom  (2),  just  a  modest  M~40,  for  No  =  Ni  and  Eq  =  Ei, 
would  bring  the  eavesdropper’s  SNR  20  dB  down  to  an  undetectable  level.  There  is  a  lot 
of  flexibility  in  picking  the  secret  rule  for  BPSK  pair  selection,  with  different  levels  of 
efficiencies  and  security.  A  keyed  EPSR  with  variable  seed  length  and  connection 
polynomial  should  be  adequate  for  most  purpose,  because  the  ciphertext  of  the  EPSR  is 
not  available  to  an  eavesdropper  from  the  MPSK  modulation  and  channel  noise  addition 
which  provides  an  automatic  probabilistic  encryption  without  any  additional  resource. 
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Figure  2:  M-ary  phase  shift  keying  signals  with  M/2  different  binary  PSK  pairs 


It  is  important  to  note  that  the  noise  Ni  the  attaeker  has  to  suffer  need  not  be  greater  than 
No,  nor  the  signal  energy  Ei  need  be  smaller  than  Eo.  Indeed,  for  any  signal-to-noise 
advantage  on  one  BPSK  pair  the  attaeker  may  have,  it  ean  be  overeome  by  a  suffieiently 
large  M  from  (2).  Typieally,  the  attacker  has  to  suffer  some  channel  noise,  at  least  the 
noise  in  his  detectors.  His  signal  level  also  cannot  go  higher  than  what  is  transmitted  by 
the  sensor. 

The  advantages  of  this  scheme  include  the  following,  hirst  of  all,  it  has  no  bandwidth 
expansion  and  can  be  used  in  conjunction  with  the  usual  spread  spectrum  technique 
useful  for  antijam  purpose.  Secondly,  it  is  much  more  truly  a  probabilistic  encryption 
compared  to  DS  spreading  in  which  each  chip  has  only  two  levels,  here  it  has  M.  Thus, 
the  above-mentioned  weaknesses  in  PN  DS  spreading  are  ameliorated.  In  particular,  in 
contrast  with  CDMA  IS-95,  the  communication  code  structure  would  not  reveal  useful 
information  on  the  EESR  output  sequence.  Eurthermore,  in  addition  to  having  high 
security  level  and  encryption  rate  at  the  signal  modulation  rate,  it  is  also  quite  efficient  in 
power  consumption  comparable  to  or  better  than  a  stream  cipher,  as  very  simple  EESR 
can  be  used.  The  disadvantages  are  that  M-ary  modulation  is  required  compared  to 
binary,  that  the  eavesdropper's  possible  signal-to-noise  level  needs  to  be  estimated  so  that 
a  sufficiently  large  M  in  (2)  is  to  be  used,  and  that  the  technique  is  new  and  has  not  been 
extensively  analyzed.  In  particular,  careful  cryptanalysis  is  needed  to  fully  quantify  its 
security  level.  In  this  project  we  propose  to  perform  such  studies  using  information 
theoretic  and  probabilistic  techniques,  with  special  emphasis  on  the  utilities  of  this 
approach  in  lightweight  encryption.  We  believe  it  is  quite  possible  that  the  technique  can 
be  proved  secure  against  every  kind  of  known  plaintext  attack. 
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Figure  3a:  Decision  regions  for  BPSK 


(a) 


L  ' 
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(b) 


Figure  3b:  Noise  vector  N  interfering  with  neighboring  BPSK 


We  have  introduced  another  novel  technique  that  significantly  lessens  the  level  of  noise 
that  is  requires  to  be  present  in  the  above  description.  In  this  approach,  the  signal  being 
transmitted  is  further  randomized  by  an  angle  Act  so  that  the  total  error  AO  +  Act  <  If . 
The  receiver  does  not  need  to  know  Act,  and  the  technique  is  to  be  called  deliberate 
signal  randomization  (DSR).  With  this  technique  we  would  sum  the  system  as  indicated 
in  Fig.  4  where  ENC  represents  a  lower  power  encryption  mechanism  such  as  that  of 
Task  A.  The  seed  key  K  determines  a  running  key  K’  through  ENC  that  executes  the 
modulation  first  via  Eig.  1  and  then  further  randomized  by  DSR.  As  a  consequence 
known  plaintext  attack  on  the  key  would  involve  an  added  search  that  is  compounded  on 
that  ENC  alone,  and  appears  to  be  experimental  in  nature.  The  use  of  this  technique  has 
now  been  proposed  for  quantum  cryptography  in  ref  [5]. 
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channel 

Figure  4:  Combined  SSi  and  Low-power  eneryption 
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IV.TASK  C:  Power  Efficient  Algorithms 


The  research  effort  has  been  focused  in  two  directions.  The  first  is  directed  towards  the 
development  of  effective  design  methodologies  and  new  design  paradigms  for  power 
conscious  systems.  The  second  is  looking  into  the  architectural  and  implementation 
issues  for  generating  power/security  tradeoffs  in  cryptosystems.  Both  systems  are  100% 
completed. 


Design  Methodologies  and  Paradigms  for  Power  Aware  Systems 

We  have  developed  methodologies  for  power  optimization  at  gate  level  through  dual 
voltages.  In  this  work,  we  propose  a  provably  good  algorithm  for  voltage  scaling  in  gate 
level  circuits.  Methodologies  for  simultaneous  voltage  scaling  and  gate  sizing  for  power 
optimization  have  also  been  developed.  These  algorithms  when  combined  with  our  novel 
delay  budgeting  procedure  can  result  in  large  decrease  in  power  dissipation  at  negligible 
performance  penalty.  At  system  level,  we  have  developed  an  optimal  power  aware  node¬ 
scheduling  algorithm  for  wireless  distributed  sensor  networks.  It  assists  in  optimizing 
power  dissipation  and  improving  system  energy/lifetime. 

The  most  important  contribution  in  the  last  few  months  has  been  the  development  of  the 
philosophy  of  predictability  applied  to  power  optimization  and  estimation.  Power 
estimation  is  extremely  hard  because  of  the  different  and  complex  parameters  involved. 
Predictability  in  such  a  scenario  is  key.  Our  latest  research  shows  that  the  accuracy  in 
prediction  is  strongly  related  to  the  design  structure  and  more  specifically  resource 
binding.  We  have  proposed  cost  functions  that  capture  the  notion  of  predictability  in 
power  at  the  resource  binding  stage.  Two  algorithms  have  been  developed  to  address  the 
predictability  optimization  problem.  The  first  algorithm  modifies  the  traditional  network 
flow  based  methodology  used  for  power  driven  resource  binding  and  optimizes  the 
predictability.  The  second  algorithm  is  based  on  the  left  edge  algorithm  and  treats  the 
input  scheduled  data  flow  graph  as  an  interval  graph.  A  complete  system  has  been 
developed  that  takes  functions  written  in  C  as  input  and  schedules  the  extracted  DFGs  in 
the  required  clock  steps.  This  scheduled  DFG  is  then  subjected  to  resource  binding  using 
the  algorithms  mentioned  above.  Experiments  on  Media-Bench  benchmark  suite  show 
that  our  methodologies  lead  to  an  average  improvement  of  81%  in  power  predictability 
with  around  13%  power  penalty  in  RTL  designs.  Some  of  the  benchmarking  C  files  used 
by  us  were  extracted  from  crypto  applications.  This  increase  in  predictability  could  be 
used  to  make  better  system  decisions  and  optimizations. 


Power  Efficient  Architectures  for  Crypto-Algorithms 

We  concentrated  on  the  development  of  power  efficient  implementations  for  crypto¬ 
algorithms.  We  used  the  Synopsys  design  flow  for  experimental  purposes.  We  have 
implemented  initial  versions  of  all  basic  blocks  in  the  reduced  -rank  filter  using  VHDL. 
However  the  digital  signal  processing  character  of  the  design  raises  a  number  of 
hardware  related  design  issues.  The  synthesizability  of  the  design  is  also  a  major  concern 
since  the  current  synthesis  tools  constrain  the  specification  language.  This  requires 
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detailed  analysis;  proper  adjustment  in  the  design  and  utilization  of  specific  techniques  to 
overcome  these  difficulties.  Our  main  goal  was  to  design  architectures  that  enable 
efficient  tradeoff  between  power  dissipation  and  security.  We  have  integrated  these 
architectures  and  implementations  with  novel  power  aware  design  methodologies  and 
paradigms  for  synthesizing  highly  power  optimized  implementations.  The  overall  results 
are  given  in  ref  [6]  -  [25]. 
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V.  TASK  D:  Image  and  Video  Processing  and  Communications 

Introduction 

The  main  objective  of  this  project  was  to  investigate  the  impact  of  video  and  imaging 
sensors  on  benchmark  and  operational  scenarios  related  to  target  tracking  in  a  wireless 
sensor  network.  This  research  was  motivated  by  the  observation  that  imaging  sensors  can 
offer  useful  and  substantially  different  information  on  a  target,  which  cannot  be  obtained 
by  other  sensing  modalities  that  have  been  considered  in  the  SensIT  program,  such  as 
acoustic,  seismic,  and  thermal.  Leveraging  on  the  state  of  the  art  in  image-based 
tracking,  the  work  was  directed  towards  improving  currently  existing  algorithms  as  well 
as  discovering  novel  methods  that  would  enhance  the  capabilities  of  existing  imaging 
technologies.  An  overview  of  the  end-to-end  imaging-sensor-based  tracking  system 
envisioned  in  this  research,  and  a  discussion  of  the  accomplishments  and  discoveries 
made  during  the  project,  are  provided  below. 

Overview 

The  basic  scenario  that  we  considered  was  that  of  detecting  a  target  as  it  enters  a  sensor 
field  composed  of  wireless  imaging  sensors,  and  continuously  tracking  the  3-D  position 
and  velocity  of  the  target  as  it  moves  through  the  field.  We  assumed  that  the  imaging 
sensors  have  been  arbitrarily  positioned  in  the  sensor  field,  and  that  they  have  arbitrary 
orientations. 

In  order  to  be  able  to  accurately  estimate  the  3-D  position  of  the  targets,  it  is  first 
necessary  to  calibrate  the  imaging  sensors.  We  considered  two  types  of  calibration 
techniques,  those  that  require  a  calibration  grid,  and  self-calibration  techniques  that  rely 
on  scene  features  for  calibration.  The  latter  are  preferable,  as  they  do  not  depend  on  an 
object  of  known  dimensions  and  position  on  the  scene.  Self-calibration  techniques 
require  the  detection  of  correspondences  between  points  in  snapshots  obtained  from  each 
camera.  This  is  particularly  difficult  when  the  cameras  are  in  a  wide  baseline 
configuration,  which  is  typical  when  an  arbitrary  sensor  placement  is  assumed.  The 
problem  is  considerably  easier  in  a  narrow-baseline  configuration  when  the  cameras  axes 
are  assumed  to  be  parallel.  Thus,  a  significant  portion  of  the  project  was  devoted  to  the 
problem  of  robust  feature  point  matching  and  camera  calibration  for  the  wide  baseline 
case. 

Another  important  problem  in  tracking  using  imaging  sensors  is  the  detection  of  moving 
targets.  It  is  generally  accepted  in  the  computer  vision  community  that  this  can  be 
accomplished  efficiently  using  background  subtraction  techniques.  We  implemented 
state-of-the-art  subtraction  techniques,  and  considered  refinements  that  increase  their 
robustness  and  accuracy. 

An  important  consideration  in  the  context  of  the  SensIT  program  is  that  the  imaging 
sensors  are  typically  part  of  a  wireless  network,  and  thus,  are  constrained  in  their  energy 
usage.  Since  imaging  sensors  in  general  consume  more  power  than  other  less  complex 
sensors,  it  is  important  to  consider  tracking  algorithms  that  optimally  select  the  set  of 
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imaging  sensors  to  be  used  for  traeking,  while  the  remaining  sensors  remain  in  an 
inaetive,  low  power  state.  In  this  eontext,  we  developed  a  novel  algorithm  that  seleets  an 
optimal  set  of  imaging  sensors  in  a  sealable  and  distributed  manner. 

The  main  goal  of  the  teehniques  we  have  studied  for  deteeting  and  traeking  targets  is  to 
minimize  the  amount  of  information  that  is  transmitted  between  the  wireless  sensors,  in 
order  to  eonserve  energy,  and  also,  to  minimize  the  probability  of  intereeption.  In  some 
eritieal  eases,  however,  it  may  be  neeessary  to  transmit  the  raw  video  data.  For  sueh 
oases,  we  developed  teehniques  that  make  effioient  use  of  transmitter  energy  in  video 
streaming  over  wireless  ohannels. 

The  basio  oomponents  of  the  system  that  were  studied  and  developed  during  this  projeot 
oan  be  summarized  as  follows: 

1 .  Wide -baseline  oamera  oalibration 

2.  Baokground  subtraotion  for  target  deteotion 

3.  Optimal  sensor  seleotion  for  multiple  sensor  traeking 

4.  Low-transmission-energy  teehniques  for  wireless  video  oommunioation 

We  now  discuss  the  techniques  in  more  detail. 

Wide  Baseline  Camera  Calibration 

As  we  saw  above,  camera  calibration  is  necessary  for  accurate  estimation  of  target 
position  and  tracking.  Camera  calibration  consists  of  estimating  the  camera  position  and 
orientation  (extrinsic  parameters)  and  the  focal  length,  skew,  aspect  ratio,  and  principal 
point  of  the  camera  (intrinsic  parameters). 

There  are  two  major  approaches  to  calibration:  grid-based  calibration,  which  introduces 
a  “foreign”  object  of  known  dimensions  into  the  scene,  and  self-calibration  which  relies 
on  constraints  (epipolar,  absolute  conic)  contained  in  the  images  themselves  to  obtain  the 
unknown  camera  parameters.  During  the  early  stages  of  the  project,  we  explored  grid- 
based  calibration  techniques.  A  working  system  that  uses  planar  circular  control  points 
was  developed,  and  was  able  to  calibrate  cameras  with  high  accuracy.  However,  in  a 
realistic  sensor  network  scenario,  self-calibration  techniques  are  obviously  preferable. 
Using  such  techniques,  cameras  can  be  calibrated  up  to  a  scale  factor.  Complete 
Euclidian  calibration  can  be  achieved  using  some  metric  information  about  the  scene, 
e.g.,  using  GPS  to  obtain  the  baseline  distance  between  cameras. 

The  main  idea  is  to  calibrate  two  or  more  cameras  using  snapshots  of  a  moving  target. 
The  target  motion  provides  several  constraints  on  the  camera  parameters.  When  using 
two  cameras,  it  can  be  shown  that  two  target  snapshots  with  8  point  correspondences  per 
snapshot  are  necessary  to  calibrate  the  two  cameras  using  a  simplified  camera  model, 
which  assumes  that  the  aspect  ratio  of  the  camera  pixels  is  known  and  that  there  is  no 
skew.  Such  assumptions  are  reasonable  for  the  current  generation  of  cameras. 
Furthermore,  the  location  of  the  principal  point  can  be  safely  assumed  to  be  at  the  center 
of  the  image  plane,  as  it  is  known  that  small  deviations  in  the  location  of  the  principal 


12 


point  do  not  significantly  affect  the  reeonstruetion  aecuraey.  More  importantly,  if  all  of 
the  intrinsie  parameters  exeept  the  foeal  lengths  are  available,  then  the  ealibration  ean  be 
performed  using  only  one  snapshot  from  the  two  cameras.  This  assumption,  whieh  in 
many  cases  is  quite  realistic,  simplifies  the  problem  eonsiderably  and  makes  the  solution 
more  robust. 

Self-ealibration  is  performed  by  exploiting  the  epipolar  eonstraint  between  images  and 
ealeulating  the  fundamental  matrix,  which  describes  pixel  correspondenees  between 
eamera  snapshots.  Intrinsie  parameters  are  obtained  from  fundamental  matriees  using  the 
Kruppa  equations,  and  extrinsie  parameters  are  obtained  by  deeomposing  an  essential 
matrix  into  its  rotational  and  translational  eomponents. 

The  wide-baseline  eamera  calibration  problem  was  taekled  in  two  major  steps.  First,  we 
developed  an  improved  feature  point  deteetion  and  eorrespondenee  algorithm  that  ean  be 
used  to  obtain  an  adequate  number  of  aeeurate  point  matehes  for  eamera  ealibration. 
Seeond,  sinee  the  point-matehing  algorithm  also  produees  a  large  proportion  of  outliers 
(ineorreetly  matehed  points),  we  designed  a  robust  eamera  ealibration  algorithm  that  ean 
eliminate  the  outliers  and  aeeurately  measure  the  eamera  parameters. 

1.  Feature  Point  Detection  and  Correspondence 

We  used  an  improved  version  of  the  Harris  eomer  detector,  which  is  commonly  used  to 
detect  feature  points.  The  Harris  comer  deteetor  deteets  points  in  the  image  around 
whieh  the  image  intensity  varies  signifieantly  in  two  direetions.  Sueh  points  usually 
oeeur  at  eomers  of  objeets  in  the  image,  and  ean  be  eonsistently  deteeted  (using  a  Harris 
detector)  in  images  of  the  objeet  taken  from  varying  viewpoints.  We  used  a  multiscale 
version  of  the  Harris  detector  in  order  to  detect  feature  points  aeross  multiple  seales.  We 
proposed  an  enhaneed  version  of  the  multiseale  Harris  deteetor  that  exploits  a  sealespaee 
filtering  teehnique  to  further  improve  the  feature  point  loealization.  The  proposed 
enhancement  seleets  points  with  high  comer  strength  at  larger  scales  and  traeks  them 
down  through  sealespaee  to  smaller  scales.  This  provides  more  accurate  feature  point 
loealization,  whieh  is  eritieal  for  eamera  ealibration. 

While  images  taken  from  eameras  in  a  wide  baseline  eonfiguration  are  generally  related 
by  a  perspective  transformation,  it  was  assumed  that  within  a  small  loeally  planar  region 
the  perspeetive  transformation  ean  be  approximated  by  an  affine  transformation.  Thus, 
onee  the  feature  points  are  deteeted,  the  image  neighborhood  around  eaeh  feature  is 
transformed  into  an  affine  invariant  form  using  the  eoncept  of  affine  Gaussian  sealespaee. 
The  affine  invariant  form  of  eaeh  point  neighborhood  is  represented  by  a  15-element 
veetor,  whieh  ean  then  be  used  to  mateh  with  veetors  obtained  from  feature  points  in  the 
image  from  another  eamera.  The  matehing  ean  be  done  using  a  Mahalanobis  distanee 
measure.  The  eorresponding  feature  points  in  the  two  eamera  images  are  then  those  that 
have  the  closest  affine  feature  vectors. 
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2.  Robust  Camera  Calibration 

We  implemented  a  robust  eamera  self-ealibration  algorithm  and  tested  it  on  indoor  and 
outdoor  scenes  using  both  manually  and  automatically  selected  feature  points.  The 
results  were  very  good,  demonstrating  the  efficacy  of  the  approach.  It  was  also  found, 
however,  that  the  process  of  fundamental  matrix  estimation  is  very  sensitive  to  noise, 
especially  when  it  results  in  feature  correspondence  mismatches,  which  in  turn  can  cause 
the  estimation  to  totally  collapse.  We  implemented  a  Random  Sample  and  Consensus 
Algorithm  (RAN SAC)  in  order  to  eliminate  outliers  caused  by  the  feature  detection 
algorithm,  and  also,  to  add  robustness  to  the  fundamental  matrix  estimation.  The 
RANSAC  tries  different  sets  of  feature  points,  chosen  randomly,  in  order  to  find  the  best 
fit  (that  is,  the  one  with  the  most  inliers).  Since  the  estimation  requires  at  least  seven 
point-correspondences,  in  order  for  RANSAC  to  work,  it  is  necessary  that  the  set  of 
feature  points  contains  at  least  seven  correct  correspondences.  Once  the  outliers  are 
detected  and  eliminated,  then  the  correct  epipolar  lines  can  be  obtained.  To  add  further 
robustness  and  precision  to  the  fundamental  matrix  estimation,  the  epipolar  constraint 
gained  from  the  initial  fundamental  matrix  estimation  can  be  utilized  to  obtain  additional 
feature  point  matches  in  a  second  stage  of  the  algorithm.  To  accomplish  this,  we  utilized 
the  epipolar  constraint  by  searching  for  a  matching  feature  point  in  the  other  image  only 
in  the  neighborhood  of  the  corresponding  epipolar  lines. 

We  also  conducted  further  testing  in  order  to  evaluate  the  robustness  of  the  algorithm 
under  non-ideal  conditions.  Our  tests  demonstrate  that  the  algorithm  is  able  to  perform 
well  even  at  high  noise  levels  (Gaussian,  PSNR  =  20dB)  and  on  compressed  images 
down  to  less  that  one  bit  per  pixel  (JPEG-standard  compression),  with  a  smooth 
degradation  as  the  noise  and  compression  increase.  Such  analysis  is  essential  in 
evaluating  the  performance  of  the  calibration  algorithm,  and  as  far  as  we  know,  has  not 
been  previously  reported  in  the  computer  vision  literature. 

Background  Subtraction 

The  following  key  considerations  were  taken  into  account  in  the  design  of  the 
background  subtraction  algorithm: 

1.  Adaptability  to  gradual  changes  in  illumination:  As  the  time  of  day  changes,  the 
lighting  conditions  of  the  system  also  change.  Therefore,  the  background  model  must 
be  updated  temporally  based  on  the  current  lighting  conditions  in  the  scene. 

2.  Robustness  to  vacillations  in  background:  In  outdoor  scenes,  trees  waving  in  the 
wind  can  cause  a  particular  pixel  in  the  image  frame  to  be  a  projection  of  a  part  of  a 
leaf  (green),  a  branch  of  the  tree  (brown),  or  the  sky  (blue).  In  all  these  cases,  the 
particular  pixel  should  be  labeled  as  background  although  its  intensity  may  differ 
significantly  between  successive  frames. 

3.  Small  training  period:  Due  to  energy  considerations  in  a  wireless  sensor  network, 
the  imaging  sensor  cannot  be  expected  to  remain  active  at  all  times.  Therefore,  the 
background  subtraction  algorithm  must  be  able  to  initialize  and  generate  a 
background  model  within  a  few  seconds. 
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We  used  a  simple,  currently  existing,  background  subtraction  technique  to  obtain  an 
initial  segmentation  of  the  background  region.  This  technique  is  based  on  maintaining  a 
non-parametric  model  for  the  background;  it  approximates  the  probability  density 
function  of  a  background  pixel  distribution  with  a  weighted  average  of  a  set  of  kernel 
functions  defined  around  sample  data  points  taken  from  the  pixel  location.  The  technique 
satisfies  the  requirements  specified  above  and  produces  reasonable  results.  However,  the 
resulting  segmentations  still  contain  some  spurious  results.  Thus,  we  refined  the 
algorithm  by  adding  spatio-temporal  continuity  constraints  to  the  background  pixel 
distribution  in  the  form  of  Markov  random  fields  (MRF).  The  final  background 
subtraction  algorithm  was  tested  using  real  data,  and  implemented  to  run  in  real  time. 

Data  Collection 

Since  adequate  video  sensor  data  were  not  available  from  the  SITEX  experiments,  the 
background  subtraction  algorithm  was  tested  using  data  collected  at  Northwestern 
University.  The  data  were  collected  for  an  outdoor  scenario  that  consisted  of  vehicles 
moving  on  a  road.  Two  USB  cameras  attached  to  laptop  computers  were  used  and  the 
cameras  were  set  to  collect  grayscale  video  at  15  frames  per  second  at  a  resolution  of 
352x288  pixels.  Included  in  the  test  cases  were  that  of  a  single  target  moving  at  20  mph, 
a  single  target  starting  at  lOmph  and  increasing  its  speed  to  20mph,  a  single  target 
starting  at  20mph,  then  stopping  and  idling  for  1  minute  and  then  accelerating  back  to 
20mph,  and  two  targets  moving  in  opposite  directions  at  20mph.  The  algorithm  was 
shown  to  work  well  in  windy,  outdoor  conditions,  and  was  shown  to  be  robust  to  changes 
in  illumination  caused  by  moving  clouds. 

Real-time  Implementation 

Once  the  algorithm  was  tested  for  accuracy,  it  was  optimized  for  use  in  real-time 
applications.  The  algorithm  was  implemented  in  C  using  video41inux  software,  which  is 
available  on  Linux  iPAQs.  It  can  run  at  10  frames/sec  on  320x240  pixel  video  data.  The 
algorithm  was  implemented  using  only  integer  arithmetic  in  a  manner  that  would  make  it 
simple  to  integrate  the  code  with  a  handheld  device  such  as  an  iPAQ. 

Multiple  Sensor  Tracking  and  Sensor  Selection 

Due  to  their  high-energy  consumption,  it  is  important  to  only  keep  a  minimum  number  of 
sensors  active  in  the  network  at  any  given  time.  Thus,  we  developed  a  method  for 
selecting  the  optimal  subset  of  imaging  sensors  to  use  for  tracking  a  moving  target  such 
that  the  energy  consumed  in  the  network  will  be  within  a  given  constraint.  The 
initialization  time  and  energy  required  for  initialization  of  the  background  subtraction 
algorithm  was  also  considered  as  a  factor  in  the  sensor  selection  algorithm.  The  sensor 
data  fusion  for  target  tracking  was  performed  using  an  unscented  Kalman  filter  (UKF) 
formulation,  which  allows  for  non-linear  dynamic  and  observation  models  and  at  the 
same  time  is  simpler  to  implement  and  more  accurate  than  an  extended  Kalman  filter. 

The  proposed  technique  selects  the  best  subset  of  sensors  based  on  an  optimization  that 
maximizes  the  information  utility  of  a  set  of  active  sensors  over  a  finite  time  window 
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subject  to  a  constraint  on  the  energy  consumption.  The  information  utility  of  a  subset  of 
sensors  is  defined  to  be  proportional  to  the  entropy  of  the  posterior  distribution  of  the 
target  state  estimate  given  that  only  that  subset  of  sensors  is  active.  The  method  is  based 
on  the  assumption  that  the  target  state  distribution  is  unimodal  and  can  be  approximated 
by  a  Gaussian  distribution.  In  the  case  of  a  Gaussian  target  state  distribution,  the 
covariance  of  the  posterior  distribution  given  measurements  from  a  particular  subset  of 
sensors  can  be  calculated  prior  to  obtaining  the  actual  measurements  from  the  sensor 
subset. 

This  method  was  implemented  in  a  simulated  target-tracking  scenario.  In  our 
simulations,  the  sensor  field  was  divided  into  a  grid  of  10m  x  10m  blocks.  Each  block 
contained  two  imaging  sensors  that  were  arbitrarily  placed  and  oriented  within  the  block. 
We  assumed  that  the  sensors  have  been  calibrated,  and  that  they  have  a  frame  rate  of 
30fps.  We  also  assumed  that  each  block  contains  one  processing  node,  which  keeps  track 
of  the  current  target  state  and  activates  or  deactivates  the  imaging  sensors  based  on  the 
sensor  selection  algorithm.  It  is  assumed  that  the  processing  nodes  can  only 
communicate  with  imaging  sensors  that  are  within  the  neighboring  blocks,  and  that  only 
the  processing  node  in  the  block  that  contains  the  target  is  active  at  any  given  time.  As 
the  target  moves  through  the  sensor  field,  the  current  processing  node  activates  the 
optimal  sensors,  obtains  measurements,  and  calculates  the  new  target  state  statistics. 
When  the  target  moves  out  of  the  block,  the  current  processing  node  hands  off  the 
tracking  task  by  activating  the  processing  node  in  the  neighboring  block  entered  by  the 
target.  Our  simulations  were  run  for  target  speeds  of  up  to  50mph,  and  comparisons  were 
made  between  the  proposed  method  for  sensor  selection  and  another  random  selection 
approach.  The  results  clearly  show  the  benefits  of  using  the  proposed  method  for  optimal 
imaging  sensor  selection.  Fig.  5  shows  an  example  realization  of  the  sensor  network,  and 
a  target  trajectory. 


Low-transmission-energy  techniques  for  communication  of  image  and  video  data 

We  developed  techniques  that  make  efficient  use  of  transmitter  energy  in  video  streaming 
over  wireless  channels.  The  goal  was  to  efficiently  utilize  transmitter  energy  while 
meeting  delay  and  video  quality  constraints.  The  key  to  the  new  techniques  is  the  joint 
selection  of  coding  parameters  and  transmitter  power  in  order  to  minimize  transmission 
energy  for  a  given  delay  and  video  quality  constraint. 

Summaries  of  our  results  are  presented  in  the  publications  [26]  -  [31]. 
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Example  Sensor  Network  and  Target  Trajectory 


X(m) 


Figure  5  Sensor  network  showing  fields  of  view  of  arbitrarily  plaeed  and  oriented 
imaging  sensors.  The  cireles  represent  the  positions  of  the  processing  nodes  in  each 
block.  An  example  target  trajectory  through  the  network  is  also  shown. 
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VI. Conclusions 


The  main  goal  of  this  project  was  to  provide  and  compare  the  security  /power  tradeoffs  of 
various  cryptographic  mechanisms  for  sensor  networks.  An  added  task  on  image 
processing  via  power  conscious  sensor  network  was  also  performed.  Several  new 
cryptographic  techniques  suited  for  low  power  applications  have  been  developed,  with 
attention  to  architectural  and  implementation  issues.  New  imaging  and  tracking 
algorithms  were  also  developed  that  further  enables  existing  techniques.  It  is  hoped  that 
these  could  be  integrated  readily  with  current  systems  in  a  realistic  application. 
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